Skip to content

fix(workbook): make the zoned and sparkline wire forms canonical-only - #774

Merged
hhimanshu merged 2 commits into
mainfrom
fix/768-workbook-schema
Jul 28, 2026
Merged

fix(workbook): make the zoned and sparkline wire forms canonical-only#774
hhimanshu merged 2 commits into
mainfrom
fix/768-workbook-schema

Conversation

@hhimanshu

@hhimanshu hhimanshu commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

closes #768

Summary

The published JSON schema had no branch for zoned and none for sparkline, so a workbook containing either serialized correctly and then failed validation against our own schema. A consumer doing the right thing was told the file was malformed.

Fixing it turned out to be less about the schema than about the wire reader.

The root cause is one thing, not five

The wire reader reuses parsers written for the formula level, where leniency is required, and inherits leniency the writer never exercises.

=SPARKLINE({1,2},{"charttype","LINE"}) must work in Sheets, so SparklineChartType::parse is case-insensitive. But from_json reusing it meant the wire accepted {"charttype":"Line"} — bytes our serializer never emits. The evidence that this was unintended sits eight lines away in the same function: the reader already rejects a non-lower-case option key.

Same story for zoned: core's parse_rfc9557 trims, because TZPARSE operates on user-typed arguments. The wire inherited that and accepted "…Z\n".

So the reader was tightened, not the schema widened, in both places. Widening instead would have documented junk as a valid document shape, which is the opposite of what a published contract is for. Core's formula-level parsers are untouched — only from_json got stricter.

⚠️ This narrows what from_json accepts

Documents with a non-canonical charttype casing, or a whitespace-padded zoned value, now fail to load. Both are unreachable from our serializer, and SPARKLINE shipped a day ago, so nothing in the wild carries either. It is safe now and gets harder every day — worth flagging as a deliberate compatibility decision rather than a silent one.

Three gaps beyond the two the issue named

  1. Both variants are reachable inside a spill anchor's array. ={TZNOW("UTC"),TZNOW("Europe/Berlin")} recalcs to an array of them, and scalarValue was missing both — so fixing only the top-level branches would have left whole documents failing.
  2. The serializer emitted bytes nothing could read. to_json succeeded on a 0- or 1-point sparkline; both the schema and from_json reject those, breaking the crate's own round-trip guarantee. Now guarded at serialization time via S::Error::custom, mirroring what Value::Array already does eight lines up. Extended to upper-case option keys and a charttype option too — all three were constructible on a public struct and all three produced unreadable bytes.
  3. The zoned pattern didn't bound field ranges. "2026-99-99T99:99:99+99:99" validated. Now bounded, including leap second :60 (which the reader accepts and normalizes) and the offset's true ±23:59.

The branches were derived from bytes, not from the enum

Values were serialized and the output read back. That caught a detail an eyeballed schema gets wrong: the bracketed zone is present for an IANA zone (…+01:00[Europe/Berlin]) and absent for a fixed offset.

The durable part is the test, not the branches

crates/workbook/tests/schema_value_variant_tests.rs validates serializer output against the schema for every Value variant, with two independent coverage gates — an exhaustive match (compile-time) and a runtime check parsing the variant list out of src/value.rs. Two hand-added branches without it would just reset the clock.

It was mutated, not asserted. Adding a hypothetical variant fails at each of three stages: compile error when the match isn't exhaustive, coverage failure when the sample is missing, validation failure when the schema branch is missing. Deleting the zoned branch fails 5 of 7 tests by name.

Every new guard was likewise reverted and confirmed to fail:

reverted fails with
charttype reader guard should have been rejected: {"charttype":"Line"…}
zoned padding guard the deserializer should have rejected {" 2026-01-01T12:00:00Z"}
writer guard should have been rejected: SparklineSpec { data: [], … }
[Tt ][Tt] the schema should have accepted 2026-01-01 12:00:00Z
field bounds the schema should have rejected 2026-99-99T99:99:99+99:99

Still inexpressible, and now documented as such

Calendar validity (2026-04-31 matches the pattern; the reader rejects it), the representable instant range (9999-01-01 matches; it's an i64 nanosecond count), and tzdb membership. The description previously named an omission among these as if it were a limitation — corrected.

Also added: consumers must use a draft 2020-12 validator. Draft-07 silently no-ops prefixItems and drops all option-pair inner validation.

How to test

cargo test -p truecalc-workbook --test schema_value_variant_tests
cargo test -p truecalc-workbook --test sparkline_value_tests

To see the original bug: check out main, construct a workbook containing a TZNOW result, serialize it, and validate against crates/workbook/schema/workbook.v1.schema.json — it fails. On this branch it passes.

Review

Test plan

  • cargo test --workspace — zero failures
  • cargo clippy --workspace -- -D warnings — exit 0
  • cargo nextest run --workspace --profile ci — exit 0
  • Every new guard mutated and confirmed non-vacuous
  • Pre-existing golden documents and the 256-case proptest pass unmodified
  • CI green

Related

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

hhimanshu and others added 2 commits July 28, 2026 16:32
…768)

`schema/workbook.v1.schema.json` had no branch for `zoned` and none for
`sparkline`, so a workbook holding either serialized correctly from Rust and
then failed validation against our own published contract -- a consumer doing
the right thing was told the file was malformed.

Both variants are reachable inside a spill anchor's array as well
(`={TZNOW("UTC"),TZNOW("Europe/Berlin")}` and `={SPARKLINE({1,2,3}),...}`
both recalc to an `array` of them), so `scalarValue` was missing them too.
Both lists gain both branches; the change is purely additive, so validation
of every existing variant is byte-for-byte unchanged.

The branches are derived from real serializer output rather than from the
enum: `zoned` carries the canonical RFC-9557 string, and `sparkline` carries
the whole parsed spec as `{charttype, data, options}` with `options` a list
of `[key, value]` pairs whose values are drawn from a narrower set than
`scalarValue` -- only number, text, boolean and empty can be a data point or
an option value.

The durable part is the test, not the two branches. Every `Value` variant now
has a representative that is serialized through `Workbook::to_json` and
validated against the committed schema, and a new variant cannot slip past it:
the variant-name `match` is exhaustive, and the sample set is checked against
the variant list read out of `src/value.rs` itself, so a variant with no
sample -- or a sample with no schema branch -- fails.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…#768)

Review follow-up. The schema branches were sized to the reader for `zoned` but
not for `charttype`, and the reader itself was more lenient than the wire was
ever meant to be. Four of the five findings are the same defect: the wire
reader reuses parsers written for the *formula* level, where leniency is
required, and inherits leniency the writer never exercises.

Reader tightened, not schema widened, in two places -- safe now because the
serializer only ever emits the canonical spelling and SPARKLINE is a day old,
so nothing in the wild carries either form:

- `charttype` is canonical lower-case on the wire. `SparklineChartType::parse`
  is ASCII case-insensitive so that `=SPARKLINE({1,2},{"charttype","LINE"})`
  evaluates, but `{"charttype":"Line"}` on the wire loaded fine and then failed
  schema validation. `parse_sparkline` now requires the canonical spelling, as
  it already did for option keys eight lines below.
- A `zoned` string may not be padded. `parse_rfc9557` trims, and trims the
  bracketed zone, for the same formula-level reason; `" ...Z"` and
  `"...[ Europe/Berlin ]"` therefore loaded and failed validation. The guard is
  at the wire boundary, so nothing relying on the formula-level trim changes.

The writer gains the guard it was missing: `SparklineSpecWire` now rejects a
spec with fewer than two data points, an upper-case option key, or a
`charttype` option, mirroring what the `Array` arm of the same `Serialize` impl
already does for its own shape rules. `SparklineSpec` is a public struct, so
these were constructible, and encoding one produced bytes that neither the
decoder nor the schema accepts -- a round-trip guarantee the crate makes and
was breaking.

The `zoned` pattern is widened where the reader is wider (a space separator, as
RFC 3339 permits) and bounded where it can be: month, day, hour, minute, second
including a leap second, and the offset's +/-23:59 range. Calendar validity
(`2026-04-31`), the representable instant range (`9999-01-01`) and tzdb
membership remain inexpressible, and the branch description now says exactly
that instead of the two vaguer claims it made before.

Also: the root description tells consumers to validate with draft 2020-12,
because a draft-07 validator silently ignores `prefixItems` and drops all inner
validation of sparkline option pairs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@hhimanshu hhimanshu self-assigned this Jul 28, 2026
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Test Coverage by Category

Category Unit Tests Google Sheets Conformance Property Cases Total
Array 42 547/547 ✓ 1,000 (2×500) 1,589
Database 35 182/182 ✓ 3,500 (7×500) 3,717
Date 369 418/418 ✓ 2,500 (5×500) 3,287
Engineering 245 886/888 ⚠ 5,500 (11×500) 6,633
Filter 11 80/80 ✓ 4,500 (9×500) 4,591
Financial 149 1,208/1,208 ✓ 2,000 (4×500) 3,357
Info 0 256/256 ✓ 4,500 (9×500) 4,756
Logical 121 263/263 ✓ 3,500 (7×500) 3,884
Lookup 69 392/392 ✓ 1,000 (2×500) 1,461
Math 536 2,006/2,006 ✓ 8,000 (16×500) 10,542
Operator 87 250/250 ✓ 7,500 (15×500) 7,837
Parser 83 92/92 ✓ 4,000 (8×500) 4,175
Query 37 37
Statistical 483 3,156/3,156 ✓ 5,000 (10×500) 8,639
Text 298 729/733 ⚠ 4,000 (8×500) 5,031
Timezone 47 47
Volatile 0 3,500 (7×500) 3,500
Web 29 56/56 ✓ 6,000 (12×500) 6,085
Total 2,897 10,521/10,527 66,000 (132×500) ~79,424

✓ = 100% passing · ⚠ = known deviation · The ~79,424 total counts formula evaluations (each conformance row and each property case = 1). GitHub Checks reports 3,741 Rust test functions: 2,897 unit + 159 property functions (shown as cases above) + 685 conformance/integration.

@hhimanshu
hhimanshu merged commit bb19752 into main Jul 28, 2026
8 checks passed
@hhimanshu
hhimanshu deleted the fix/768-workbook-schema branch July 28, 2026 06:02
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 28, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(workbook): the JSON schema is missing branches for non-scalar value variants

1 participant